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[57] 



ABSTRACT 



A method and device for preprocessing streams of encoded 
data (e.g. compressed in accordance with an MPEG 
standard) to permit a decoder to decode the streams, back- 
to-back (i.e., one stream immediately following another), 
without being reset and without producing video artifacts. 
The present invention includes verifying that a multiplexed 
stream complies with an encoding standard and preprocess- 
ing packeis of the packelued encoded video sequence such 
that no video artifacts are produced when a video decoder 
decodes an adjacent encoded video sequence. 

17 Claims, 16 Drawing Sheets 



Docket no. 2134 



-1- 



EXHIBIT C 




U.S. Patent Nov. 17, 1998 Sheet 1 of 16 5,838,678 



0 



5 



Docket no. 2134 



-2- 



EXHIBIT C 



U.S. Patent 



Nov. 17, 1998 



Sheet 2 of 16 



5,838,678 




00 

O- 



o- 



3- 



o- 



CO 

2 

Xi 

5 

cd 

cd 

CO 
W 




s -s 




0) X 

^ o ^ 



1$ 



o 
o 




W g 3 c 
Oh *a 



■3 M 



cd 



cd 

o 

bO u 

'C n 
o o 



o 

o 




CO 3 

flu a 



CO 

c 




CM 



G 

O 

CO 

C 

CO «J 

s ^ 



03 

o 

*> CO U 

a flu U 



c <g 
2 -5 

•i-l r^> 

*o o 

cd u 



Q S £ 



u 

C/J tj 



Pi 
u 

w 



CO so 



CO 



22 

co co 



§3 

co i) 2 



6 
o 



c 3 
CO «) 2 M 

w a o r 



co 




O 

^ 13 Tj 

cd 4) JJ 



s 

cd 

CO > cd 
W "G cd 

flu a ^3 



40 



CO 



0 



< 

o 

a; 
a. 



Docket no. 2134 



-3- 



EXHIBIT C 



U.S. Patent Nov. 17, 1998 Sheet 3 of 16 



t 



5,838,678 




Docket no. 2134 



-4- 



EXHIBIT C 



• t 

U.S. Patent Nov. 17, 1998 Sheet 4 of 16 5,838,678 





No. of bits 


Mnemonic 


next_start_codeO 






sequence headerO 302 






if (nextbitsO = extension_start _code) { 






sequence extensionO 304 






do{ 






extension and user data(0) 306 






do{ 






if (nextbitsO = group_start_code) { 






308 


group jrfjrictures_headerO 






310 


extension_and_user_data( 1) 






} 






312 


pictureJieadeiO 






314 


picture_coding_extensionO 






316 


extensions_and_user_data(2) 






318 


picture dataO 






} while ( (nextbitsO=picture_start_code) II 






(nextbitsO = group_start_code) ) 






if (nextbitsO! = sequence_end_code) { 






sequencejieaderf) 






sequence_extensionO 






} 






} while(nextbitsO!=sequence_end_code) 






} else { 






/♦ISO/EEC 11 172-2 */ 






} 






sequence end code 320 


32 


bslbf 


} 
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pictureJieaderO { 


312 


No. of bits 


Mnemonic 


picture_start_code 


502 


32 


bslbf 


teraporal__reference 


504 


10 


uimsbf 


oicture coding tvne 


3 


uimsbf 


vbv_delay 


16 


uimsbf 


if(picture_coding_type = 2 II picture_coding_type = 3) { 






full_j>el_forward_vector 


1 




forward _f_code 


3 


uimsbf 


} 






if (picture_coding_type = 3) ( 






full_pel_backward_vector 


1 




backward_f_code 


3 


uimsbf 


} 






while (nextbitsO = ' 1") { 






extra_bit_picture 1* with the value '1' */ 


1 


uimsbf 


extra_information_picture 


8 




} 






extra_bit_picture /* with the value '0' */ 


1 


uimsbf 


next_start_codeO 






} 







FIG. 5 (PRIOR ART) 



group of pictures headerO { 308 


No. of bits 


Mnemonic 


group_start__code — v — -604 


32 


bslbf 


time_code 


25 


bslbf 


closed_gop - — 602 


1 


uimsbf 


broken Jink 
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uimsbf 


next_start_code0 
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METHOD AND DEVICE FOR 
PREPROCESSING STREAMS OF ENCODED 
DATA TO FACILITATE DECODING 
STREAMS BACK-TO BACK 

BACKGROUND OF THE INVENTION 5 
a. Field of the Invention 

10 



15 

b. Related Art 

The MPEG standard focuses on the encoding and trans- 
port of video and audio data. Tn general, the MPEG .standard 
uses compression algorithms such that video aod audio data 2Q 
may be more efficiently stored and communicated. 

The International Organization for Standardization (or the 
Organization International Dc Normalisation) (hereinafter 
referred to as "the ISO/IEC) has produced the MPEG II 
standard for the coding of moving pictures and associated ^ 
audio. This standard is set forth in four documents. The 
document ISO/TEC 13818-1 (systems) specifies the system 
coding of the specification. It defines a multiplexed structure 
for combining audio and video data and means of repre- 
senting the timing information needed to replay synchro- 30 
nized sequences of the audio and video data in real-time. The 
document ISO/IEC 13818-2 (video) specifies the coded 
representation of video data and the decoding process 
required to reconstruct pictures* The document ISO/IEC 
13818-3 (audio) specifies the coded representation of audio 35 
data and the decoding process required to reconstruct the 
audio data. Lastly, the document ISO/IEC 13818-4 
(conformance) specifies procedures for determining (he 
characteristics of coded bitstreams and for testing compli- 
ance with the requirements set forth in the ISO/EEC docu- ^ 
ments 13818-1, 13818-2, and 13818-3. These four 
documents, hereinafter referred to, collectively, as "the 
MPEG II standard" or simply "the MPEG standard", are 
incorporated herein by reference. 

45 
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FIG. 7 is a high level block schematic showing a system 
for encoding, communicating, and decoding video and audio 
data in accordance with the MPEG H standard. 

20 



25 



30 



35 



40 

The transport stream multiplexer 712 multiplexes the 
encoded audio and video packets to form a transport stream 
100 and provides the transport stream 100 to communica- 
45 tions link 722. At a remote end of the communications link 
722, a transport stream demultiplexer 724 receives the 
multiplexed transport stream 100. 



50 



55 



60 



As mentioned above, transport streams 100 permit one or 
more programs with one or more independent time bases to 
65 be combined into a single stream. That is, a transport stream 
100 may include a first program and a second program. In 
presently contemplated systems, both the video decoder 730 
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and the audio decoder 734 must be reset before decoding a 
next program, for reasons which will be explained below. 
Thus, for example, in such systems there must be a temporal 
gap (e.g., one second) between the decoding of the first and 
second programs to permit the video and audio decoders 730 
and 734, respectively, to be reset. This temporal gap pre- 
cludes the playing of the second program directly following 
the first program. Moreover, it is difficult to determine when 
one program ends and another begins In real-time. Thus, a 
method and/or a device is needed to permit more than one 
program to be played (i.e., decoded) back-to-back. The 
method and/or device should also overcome, or avoid, the 
difficulties of cktennirring program boundaries in real-time. 
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an audio sequence and a video decoder can, without being 
reset, decode the encoded video sequence to produce a video 
sequence. The method of the present invention (i) verifies 
that the multiplexed stream complies with an encoding 
standard, (ii) preprocesses packets of the packetized, 
encoded, video sequence such that oo video artifacts are 
produced when the video decoder decodes an adjacent 
encoded video sequence, and (iii) preprocesses the packets 
of the packetized, encoded, audio data sequence such that its 
start time is within a first predetermined time of the start 
time of the video sequence and its temporal length is within 
a second predetermined time of the temporal length of the 
video sequence. 

The step of preprocessing the packets of the packetized, 
encoded, video sequence preferably includes (i) deleting any 
video frames that cannot be decoded if video frames of the 
video sequence arc not temporally correct, and (ii) deleting 
any video frames following a code indicating an end of the 
encoded video sequence. The step of preprocessing the 
packets of the packetized, encoded, audio sequence prefer- 
ably includes (i) removing any partial audio frames, (ii) 
adjusting (Le., adding or deleting) the number of audio 
frames, if necessary, such that the audio and video sequences 
start within the first predetermined time, and (iii) adjusting 
(i.e., adding or deleting) the number of audio frames, if 
necessary, such that the temporal lengths of the audio and 
video sequences are within the second predetermined time. 
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40 



Thus, as can be appreciated from the above discussion of 
MPEG and MPEG U video encoding, the video decoder 730 
often needs past and future frames to decode a picture 
(B-Pkturc) frame. If the last temporal (displayed) frame of 
a first program is used in decoding a first temporal 
(displayed) frame of a second program, or if a partial video 
frame is used, the output of the video decoder 730 will have 
been improperly decoded, disadvantageous! y causing video 
artifacts. Thus, as discussed above, the decoders must be 
reset between programs in known systems. 



SUMMARY OF TIIE INVHVTION 



Specifically, the present invention provides a method for 
preprocessing multiplexed streams of packets of packetized, 65 . 
encoded, audio and video sequences such that an audio 
decoder can decode the encoded audio sequence to produce 
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8 

FIG. 6 illustrates the structure of a group of pictures 
header of the MPEG II video sequence of FIGS. 3a and 36. 

FIG. 7 illustrates an encoding, transmission, and decoding 
system envisioned by MPEG II. 
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BRIEF DESCRIPTION OF TIIE DRAWINGS 



FIG. 2 is a diagram which illustrates the syntax of an ( 
. MPEG II PES packet 

FIGS. 3a and 36 illustrate the organization of an MPEG 
II video sequence. 

FIG. 4 illustrates (he structure of a sequence header of the T 
MPEG II video sequence of FIGS. 3a and 36. i 

FIG. 5 illustrates the structure of a picture header of the 
MPEG II video sequence of FIGS. 3a and 36. 
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